Posts Tagged ‘ Augmented Reality

The ARtefactKit – Heritage Jam 2017 Winner

Somehow the Heritage Jam run by the University of York has come round around again and gone. As I outlined in this post the Heritage Jam is an opportunity for people to get together and create new heritage visualisations relating to a specific theme. The theme this year was ‘Bones of Our Past’ – as I couldn’t be there in person, I decided to go ahead and put something together for the online competition.

It turns out my entry won first place! I built something that I have wanted to experiment with for a quite a while – an Augmented Reality application that allows you to take a real artefact (in this case a bone) and compare it to a virtual reference collection. By using your phone you can augment a ‘virtual lab’ onto your kitchen table and then use the app to call up a number of different bones from different animals until you can find one that matches.

The AR aspect of it adds something more to the ‘normal’ online virtual reference collections – by allowing you to augment the models in the correct scale in front of you and then twist and turn each one side by side.

In addition, as I am interested in multi-sensory things, I also added in the sounds and smells of the animals – as well as a virtual portal into a 360 degree video of a deer herd in action.

Finally, it has a link through to a set of Open Data from Open Context showing where else in the world similar types of bones have been found.

You can watch the visualisation here:

Please check out the full visualisation and explanation here: http://www.heritagejam.org/new-blog/2017/10/27/the-artefactkit-stu-eve

As with all of these ‘jam’ projects, the app is just a prototype and is quite messy in terms of overall look and feel – but I think it has potential to be quite useful. Now I just need some funding!

ARkit and Archaeology – Hougoumont Farm, Waterloo

For the last 3 years I have had the absolute privilege of being one of the archaeological directors of the current excavations of the battlefield of Waterloo. As part of the incredible project Waterloo Uncovered (http://www.waterloouncovered.com) – we have been taking wounded serving and veteran soldiers, students and professional archaeologists  to the battlefield to conduct the first major systematic excavation of parts of the battlefield that shaped the history of Europe in 1815.

We only have two weeks in the field each year, which means there is not a lot of time to doing anything but excavate, record and backfill (see the dig diaries and videos of all we got up to here). However, this year I managed to find the final afternoon to play with the new Apple ARkit and see what potential there is for archaeological sites.

The short answer is that there is a lot of potential! I have discussed Augmented Reality and archaeology to the nth degree on this blog and in other places (see here for a round-up) – but with the beta release of ARkit as an integrated part in iOS11, Apple may have provided the key to making AR more accessible and easier to deploy. I tried out two experiments using some of the data we have accrued over the excavations. Sadly I didn’t have any time to finesse the apps – but hopefully they should give a hint of what could be done given more time and money  (ahem, any prospective post-doc funders – my contact details are on the right).

Exploring the lost gardens of Hougoumont

The first video shows a very early experiment in visualising the lost gardens of Hougoumont. The farm and gardens at Hougoumont were famously defended by the Allied forces during the battle of Waterloo (18th June 1815). Hougoumont at the time was rather fancy, with a chateau building, large farms buildings and also a formal walled garden, laid out in the Flemish style. One of the participants this year, WO2 Rachel Willis, is currently in the process of leaving the army and studying horticulture at the Royal Horticultural Society. She was very excited to look at the garden and to see if it was possible to recreate the layout – and perhaps even at some point start replanting the garden. To that end she launched herself into the written accounts and contemporary drawings of Hougoumont and we visited a local garden that was set out in a similar fashion. Rachel is in the process of colouring and drawing a series of Charlie Dimmock style renditions of the garden plans for us to work from – but more on that in the future.

Similar gardens at Gaasbeek Castle

Extract from Wm. Siborne’s survey of the gardens at Hougoumont

As a very first stab at seeing what we might be able to do in the future, I quickly loaded up one of Rachel’s first sketches into Unity and put a few bushes and a covered walkway in. I then did some ARkit magic mainly by following tutorials here, here, and here. Bear in mind that at the time of writing, ARkit is in beta testing, which means you need to install Xcode Beta, sign up for and install the iOS 11 beta program for the iPhone and also run the latest beta version of Unity. It is firmly at the bleeding edge and not for the faint-hearted! However, those tutorial links should get you through fine and we should only have to wait a few months and it will be publicly released.  The results of the garden experiment are below:

As can be seen, the ARkit makes it very simple to place objects directly into the landscape OUTSIDE – something that has previously only really been possible reliably using a marker-based AR plugin (such as Vuforia). Being able to reliably place AR objects outside (in bright sunshine) has been somewhat of a holy grail for archaeologists, as unsurprisingly we often work outside.  I decided to use a ‘portal’ approach to display the AR content, as I think for the time being it gives the impression of looking through into the past – and gives an understandable frame to the AR content. More practically, it also means it is harder to see the fudged edges where the AR content doesn’t quite line up with the real world! It needs a lot of work to tidy up and make more pretty, but not bad for the first attempt – and the potential for using this system for archaeological reconstructions goes without saying! Of course as it is native in iOS and there is a Unity plugin, it will fit nicely with the smell and sound aspects of the embodied GIS – see the garden, hear the bees and smell the flowers!

Visualising Old Excavation Trenches

Another problem we archaeologists have is that it is very dangerous to leave big holes open all over the place, especially in places frequented by tourists and the public like Hougoumont. However, ARkit might be able to help us out there. This video shows this year’s backfilled trenches at Hougoumont (very neatly done, but you can just still see the slightly darker patches of the re-laid wood chip).

Using the same idea of the portal into the garden, I have overlaid the 3D model one of our previous trenches in its correct geographic location and scale, allowing you to virtually re-excavate the trench and see the foundations of the buildings underneath, along with a culverted drain that we found in 2016. It lines up very well with the rest of the buildings in the courtyard and will certainly help with understanding the further foundation remains we uncovered in 2017. Again, it needs texturing, cleaning and bit of lighting, but this has massive potential as a tool for archaeologists in the field, as we can now overlay any type of geolocated information into the real world. This might be geophysical data, find scatter plots or, as I have shown, 3D models of the trenches themselves.

These are just very initial experiments, but I for one am looking forward to seeing where this all goes. Watch this space!

 

Pokémon Go Home – Why Pokémon is not what the heritage sector really needs

Gently I edged toward the beast. It had 4 long semi-transparent wings, the same length as its tube-like body. The body was iridescent in the light, changing colour through blue to green.  Its face was incredibly ugly… a mixture of large bug-like eyes on the side of its head and a gaping mouth filled with prehistoric fangs. It fluttered its wings gently in the breeze, as it cleaned itself by rubbing its long spindly white legs all over its body. I reached down quietly in readiness to capture it, brought my phone up to look through the camera feed. Just as I was about to swipe the screen it got alarmed, its wings became a blur of movement and it took off – flitting away out of sight. I attempted to chase it – but it was gone in a blink of an eye. Another two years would pass before my friend and I got another tip-off about the location of the quasi-mythical beast.

I could be talking about my hunt for the Yanma, the large dragonfly, famous for being able to see in all directions at once and having such extreme wing-speed that it can shatter glass. Except of course I am not talking about my hunt for a Pokémon – I am talking about the day I went out with a very good friend of mine hunting in the Fens for the white-legged damselfly. We didn’t manage to capture a picture of it that day, but we did briefly see it alight on a leaf, which was more than good enough.

White-legged Damselfly by Philipp Weigell

White-legged Damselfly by Philipp Weigell (picture taken by Philipp Weigell) [CC BY 3.0 (http://creativecommons.org/licenses/by/3.0)], via Wikimedia Commons

Pokémon Go fever has been sweeping seemingly everyone in the last couple of weeks. It has been hailed as a saviour to stop idle kids sitting in front of their computer screens, the herald of mass-adoption of mainstream Augmented Reality, and the re-invigorator of an interest in heritage and museum sites.

As someone who has done quite a bit of research on AR and heritage, I would like to wade into the discussion here and question a few things. To go someway toward mitigating the risk of sounding like a grumpy old man who doesn’t understand ‘kids today’ (too late!) –  I would like to point out that I have 3 boys of my own, play all sorts of different games on both smartphone and computer and I once even bought a girlfriend a pair of Pikachu pyjamas. I understand the nostalgic call of Pokémon, the excitement of the treasure hunt and the lure of the screen over the real world.

Pokémon Go as an AR revolution

First I would like to tackle the pedantic bit. Pokémon Go is not really an AR game. The AR element of it (overlaying the Pokémon onto your video feed) is (as someone else has already said) the most basic of AR possible. So much so that it can’t really be called AR at all. There is no interaction with or reading of the real surroundings, the Pokémon don’t hide behind the cooker or pop out from behind the Museum gates. You could be standing in your toilet or at the edge of the Grand Canyon and the Pokémon would still be badly superimposed, via a simple geo-location, onto the video feed. Even the Snapchat AR which superimposes silly things on people’s faces is more AR – as at least it is doing some kind of recognition of the real world (in that case facial recognition).

Calling Pokémon Go an AR revolution is doing a disservice to the potential and power of AR for integrating with the real world. AR has so much more potential than this. Pokémon Go is a locative game, not true AR.

Pokémon Go gets kids outdoors

What’s not to like about this? Even a cynical old git like me surely can’t complain about kids (and adults) getting away from their screens and going outside. Except, of course, they are not getting away from their screens. In fact it is probably worse – by taking the screen outside and searching for Pokémon through it they are not even really taking part in the outside world. The outside world is being entirely mediated through the screen – a small rectangular box guides your every movement. The alternate reality provided by the smartphone is so beguiling that there are people falling in canals, crashing into police cars and even plummeting off the edge of cliffs whilst playing the game. Clearly even though they are outside, they are oblivious to the world around them.

Do we really live in a world where kids can’t be bothered to get off the sofa and go outside without taking a screen or a game with them? What kind of world is this? What is it going to become if the next generation take this as normal? Why is it that hunting for a Squirtle is seen as the utmost of cool – but following a tip-off about the location of a Spoonbill or standing on the end of train platform hunting trains is seen as the ultimate in nerdiness?

I’m not sure I can really see the logic. I guess that Pokémon Go is the epitome of easy and quick satisfaction. Sure you may have to travel a little, to get to a place to capture the computer-generated critter – but when you arrive you don’t have to wait and watch and hope that you glimpse a sight of it. You don’t have to be silent and scan the sky with your binoculars and be PATIENT. If someone has said that the Charmander is there, it is pretty much guaranteed that if you go to those Lat Long coordinates you will find it. Bird-watching is not the same. You can go back to the same hide for days and days and perhaps not spot what you are looking for. It may even be there, but you might not have done quite enough research to differentiate the colour of the wing flash. It is not quick or easy, and because of that it is surely more ultimately satisfying.

Pokémon Go brings all the kids to the (Archaeological) Yard

This then brings me to the final point – Pokémon Go as a way to get people more engaged with heritage sites. We have seen this before, museums and heritage sites jumping on trendy locative game bandwagons to get more people to come to their sites (Andy Dufton and I wrote about this with Foursquare a few years ago). I think it may be a little early to say whether or not this is really going to be a big thing. We will need to see stats on the increases in ticket sales to show that the Pokéhunters are not just going to the museum car park. And if they are paying the ticket price and entering the site, how much are they actually engaging with the archaeology?

Charmanders in the BM

Terry Brock is also hopeful about this:

Terry Brock

As Andrew Reinhard’s archaeogaming foray shows, there is the potential for providing extra contextual information at the ‘cultural’ Pokéstops. However a quick look at his example of the Pokéstop at his local Washington memorial shows only the information that is on the plaque of the monument itself – but then you would have to look away from your screen to read that.

Route of Washington’s March monument (taken from the Archaeogaming blog by Andrew Reinhard) – https://archaeogaming.com/2016/07/09/pokemon-go-archaeogaming/

So let us stand back a little and think about what all this means. I’ve concentrated recently on creating ways for people to use Augmented Reality to engage with, explore and understand heritage sites (take a browse around my website to see some examples). The key for me is that by someone visiting the site physically they can engage both their body AND their mind simultaneously. The AR I use is exclusively made to facilitate this, to show hidden elements of the site, to waft unexpected smells to make you THINK about the space in different ways, to play sounds that have some kind of relevance to what happened in that location in the past.

A visit to an archaeological site by a Pokéhunter is the antithesis of this. When a Pokéhunter arrives at a site (drawn by the lure of a rich Pokéstop) they are in the classic state of Cartesian disconnect. Their body may be there, but their mind is far away, thinking of the next Pokéstop or the phone notification that just came through from their mate about a rare [insert rare Pokémon name here] up the road.

You only have to look at this tweet to see the effects of this:

https://twitter.com/ohmycrayon/status/751778120647180288

This girl is at STONEHENGE, for crying out loud. Instead of taking an interest in how the stones were put up, how they fit into the surrounding landscape, what actually happened in and around them, and, crucially, how the experience of actually being there makes her feel – she is chasing an Eevee. She herself admits her attention is “so divided right now”. If this is happening at one of Britain’s most iconic and engaging monuments – what does it mean for other heritage sites? This girl’s mind is clearly not in the same place as her body. She is engaged in two separate realities, linked only by coordinates on a Google Map. Using Pokémon Go to get bums on seats and through the ticket barriers might be good for sales, but at what cost? If it really takes a Squirtle to get our youth (and adults) to go to a heritage site, then we are doing something very wrong.

What about the Real World?

I’m sorry this post has been rather despairing. I am getting increasingly sad for the state of the world, where people go head over heels hunting virtual creatures, while the real incredible biodiversity is ignored, built over and marginalised. Instead of re-wilding the world with animals, insects, plants and birds we are enchanted by the opposite: introducing the computer and virtual creatures into our diminishing natural and cultural spaces. How can it be that I am in the minority for being bewitched by the hunt for the white-legged damselfly, a beautiful, crazy, prehistoric looking creature – while the vast majority of people are instead happy to jump in their cars, park in the car park of the local baptist church and stare into their phones flicking imaginary red balls at imaginary creatures?

I haven’t even touched on the inevitable monetisation of all this, how long will it be until the big museums have to pay Niantic loads of money to host an incredibly rare Pokéstop and the smaller sites (that are actually crying out for visitors) will be priced out of the Pokémarket?

If you really can’t get your kids (or yourself) out to a heritage site without gamifying it by chasing animals, why not go and find that pair of peregrine falcons roosting in the local church steeple? Or go newt-hunting in your local historic ponds? Perhaps try to spot a red kite above the prehistoric landscape of Dartmoor? You could even use this map of rare bird sightings around the country to plan a day out birding and visiting nearby heritage sites.

But please please please – leave your smartphone behind.

CAAUK 2016 – Embodied GIS and applied Multi-Sensory Archaeology

I recently attended the CAAUK 2016 meeting in Leicester, a great couple of days with a few really interesting papers.

As usual, the rather excellent Dougs Rocks-Macqueen was on hand to record the talks. His videos can be found here – he records all sorts of diverse archaeological conferences, so it is well worth clicking the subscribe button on his account.

In case anyone is interested, I have embedded the video of my talk below – where I discuss the Embodied GIS, using examples from my previous research including Voices Recognition and the Dead Man’s Nose.

Guest Blog on ASOR

I have just submitted a guest blog post on the American Schools of Oriental Research (ASOR) blog for their ongoing special series on Archaeology in the Digital Age. It’s an introduction to Augmented Reality for Archaeology and also includes some sneak peeks of the results of some of my own AR fieldwork on Bodmin Moor. The original post can be found at http://asorblog.org/?p=4707.

Archaeology, GIS and Smell (and Arduinos)

I have had quite few requests for a continuation of my how-to series, for getting GIS data into an augmented reality environment and for creating an embodied GIS. I promise I will get back to the how-tos very soon, but first I wanted to share something else that I have been experimenting with.

Most augmented reality applications currently on the market concentrate on visual cues for the AR experience, overlaying things on a video feed, etc. There are not a lot that I have found that create or play with smells – and yet smell is one of the most emotive senses. In the presentation of archaeology this has been long known and the infamous and varied smells of the Jorvik Centre are a classic example of smell helping to create a scene. The main reason for this lack of experimentation with smells is presumably the delivery device. AR is quite easy to achieve now within the visual realm mainly because every smartphone has a video screen and camera. However, not every phone has a smell chamber – never mind one that can create the raft of different smells that would be needed to augment an archaeological experience. As a first stab at rectifying this, then, I present the Dead Man’s Nose:

The Dead Man’s Nose

The Dead Man’s Nose (DMN) is a very early prototype of a smell delivery device that wafts certain smells gently into your nose based on your location. The hardware is built using an Arduino microcontroller and some cheap computer parts along with any scent of your choice. The software is a very simple webserver that can be accessed via WiFi and ‘fire off’ smells via the webserver’s querystring. This means that it can easily be fired by Unity3D (or any other software that can access a webpage) – so it fits very nicely into my embodied GIS setup.

How does it work?

This little ‘maker hack’ takes it inspiration from projects such as ‘My TV Stinks‘, ‘The Smell of Success‘ and Mint Foundry’s ‘Olly‘. Essentially, I followed the instructions for building an Olly (without the 3D housing) and instead of using an Ethernet shield for the Arduino – I connected it to a WiFi shield and from there joined it to an ad-hoc WiFi network created by my Macbook. With the Macbook, iPad and the DMN on the same network it is very easy to send a message to the DMN from within the Unity gaming engine. As the iPad running the Unity application knows where I am in the world (see the previous blog) it means that I can fire off smells according to coordinates (or areas) defined in a GIS layer. Therefore, if I have an accurate ‘smellscape’ modeled in GIS, I can deploy that smellscape into the real world and augment the smells in the same way that I can augment the visual elements of the GIS data.  The code is very simple for both ends, I am just using the a slightly adjusted sample WiFi shield code on the Arduino end and a small script on the Unity end that pings the webserver when the ‘player’ moves into a certain place on the landscape. When the webserver is pinged, it starts the fan and that wafts the smell around. From a relatively simple setup, it provides the possibility of a very rich experience when using the embodied GIS.

A Field Test

The first thing to do was to find the smells to actually augment using the Dead Man’s Nose. It turns out there are a lot of different places to buy scents, but luckily in this case archaeologists came to the rescue – an article in the excellent Summer 2012 edition of Love Archaeology e-zine pointed me to the website of Dale Air who have over 300 aromas ranging from the mundane (Crusty Bread) to the completely weird (Dragon’s Breath). I purchased a set of samples (Barbeque, Dirty Linen, Woodsmoke, Farmyard, among others) and was ready to go. I was quite surprised, but they do actually smell pretty much as described, especially the Dirty Linen.

As I was just experimenting, the housing for the DMN was very simple (a cardboard box) and there was only one choice of smell and that was sellotaped to the outside of the box…

The Dead Man’s Nose, in a box with a BBQ scent attached

The prototype was then loaded into a bag (in this case a simple camera bag), which was slung around my neck. I popped the top of the BBQ scent open and then whenever the fan started whirring the sweet, slightly acrid smell of Barbequing meat was gently wafted to my nostrils.

The Dead Man’s Nose in a nosebag, ready to go

Using my embodied GIS of the roundhouses on Leskernick Hill, Bodmin Moor, I set the DMN to fire off a smell of lovely Barbeque whenever I got within 20m of a roundhouse. I set the fan to run slowly at first and get faster as I got closer to the ‘source’ of the smell. The DMN performed admirably, as I walked within range of the houses I heard the tell-tale whirr of the fan and the next moment I had the lovely scent of cooking ribs. Future models will allow for more than one smell at a time (I just need a couple more computer fans) and also a better housing, a bit of 3D printing is in order!

Now I can use the iPad to view the roundhouses overlaid onto the video feed, plug in my headphones and hear 3D sounds that get louder or quieter depending on where I am in the settlement and also I can augment different smells as I walk around. Not only can I walk around the modern day Bronze Age landscape and see the augmented roundhouses, hear the Bronze Age sheep in the distance, I can also smell the fires burning and the dinner cooking as I get closer to the village….

If there is interest I can put together a how-to for creating the system, but for now I am going to carry on experimenting with it – to refine the delivery and the housing and to clean up the code a little bit.

Future of Conference Posters?

Last month I entered a poster into the UCL Graduate School Poster Competition and was lucky enough to win first prize. I find conference posters a bit of a strange animal. The poster session always seems to take place over lunchtime or the coffee break and more often than not the person who made the poster isn’t around to talk you through it. You are then usually left with a poster that has masses of text, that either has too much detail or not enough, and the whole thing can get quickly boring.

I wanted to challenge this a little bit, and as my poster subject was my work with AR, I was provided with the perfect opportunity. The poster was a pretty simple (but hopefully striking) design, a pair of old school binoculars looking at some rocks on Leskernick Hill, Bodmin Moor. The area within the binoculars shows some roundhouses  – giving the impression that looking through the bins reveals the ancient landscape.

My winning poster

I tried to keep the text to a bare minimum so that the poster was dominated by the binoculars. However, this being an AR project, there was a bit of a twist. Using the Junaio API I augmented the poster with a video that overlaid the whole thing when viewed through a smartphone or tablet. The video showed the binoculars moving around the poster, revealing more of the roundhouses.

I am increasingly finding that the best way to explain AR is to give someone an example of it. It was a bit of a gamble, as in order to see the AR content the viewer needed to have a smartphone, have an app to scan the QR code on the poster and have good enough internet access to install and run the Junaio app. The main judge of the competition wasn’t at the prize-giving, so I didn’t get any feedback or a chance to ask if they had seen the AR content, but they awarded it first prize so I hope they did!

I am of course not the first person to use AR in a poster, but I am sure that it will become a lot more popular as it really is an excellent way of adding content to a poster, without being too intrusive. I guess at the moment it could be seen as being a little gimmicky, however this isn’t all that bad when trying to attract people to your poster and your research. One of the important things to remember though is that the poster needs to be able to stand on it’s own without the AR content, as it is quite an ask at the moment to get people to download an app on their phone just to learn more about your research.

The process of adding the content via the Junaio app also wasn’t quite as easy as I had hoped, mainly because the video itself had to be made into a 3D object and be of a very low quality and in a special .3g2 format to enable it to be delivered fast to a mobile device. You immediately lose your audience if they have to wait 2 minutes for your content to download and the .3g2 format was specifically designed to look ok on a smartphone screen and be small enough to download quickly. However, as you can see from the video above, the quality is pretty poor. I created the animation using 3D Studio Max, and then rendered it out to a number of tiffs. I then used ffmpeg to render the tiffs to a video and encoded it into the .3g2 format. The Junaio developer website has instructions for how to do all of this, but it is not really for the faint of heart.  Junaio provides a number of sample PHP scripts that can be run on your own server to deliver the content, and their trouble-shooting process is really excellent. So if you have your own webserver and are happy with tweaking some scripts then you can do some really quite nice stuff. I should note that they also have a simple upload interface for creating simple AR ‘channels’ which is a great way of quickly getting things up there – but doesn’t allow you to have total control or upload videos. But, if you want to pop a simple 3D model on your conference poster, then the Junaio Channel Creator is the app for you! The other thing to remember if you want to augment your own conference poster, is that the channels can take up to a week to be approved by Junaio, so you can’t leave it all to the last minute!

I suspect we will be seeing many more AR-enabled conference posters, particularly as AR booklets, magazines and museum guides are becoming more popular. One can envisage holographic type projections of people standing beside their posters talking the viewers through it, or interactive posters where the content changes depending on what and where you touch it. As I keep coming back to on this blog, it is the melding of the paper with the digital that I find so fascinating about AR, the ability to re-purpose old ideas (such as the conference poster) and breathe new life into the concept – but without losing the original purpose and feel of the thing itself. The design of the paper poster stands on its own (for better or worse!) and the AR content just gives the creator the chance to provide further information and give the viewer that extra dimension into their research.

Augmenting a Roman Fort

The following video shows something that I have been working on as a prototype for a larger landscape AR project.

As you can see, by using the Qualcomm AR SDK and Unity3D it is possible to augment some quite complex virtual objects and information onto the model Roman fort. I really like this application, as all I have done is take a book that you can buy at any heritage site (in the UK at least) and simply changed the baseboard design so that the extra content can be experienced. Obviously there was quite a lot of coding behind the scenes in the app and 3D modelling, but from a user point of view the AR content is very easy to see – simply print out the new baseboard, stick it on and load up the app.

For me that is one of the beautiful things about AR, you still have the real world, you still have the real fort that you have made and can play with it whether or not you have an iPad or Android tablet or what-have-you. All the AR does is augment that experience and allow you to play around with virtual soldiers or peasants or horses instead of using static model ones. It also opens up all sorts of possibilities for adding explanations of building types, a view into the day-to-day activities in a fort, or even for telling stories and acting out historical scenarios.

The relative ease of the deployment of the system (now that I have the code for the app figured out!) means this type of approach could be rolled out in all sorts of different situations. Some of my favourite things in museums, for instance, are the old-school dioramas and scale-models. The skill and craftsmanship of the original model will remain, but it could be augmented by the use of the app – and made to come alive.

The model of Housesteads fort in the Housesteads museum

The same is true of modern day prototyping models or architectural models. As humans we are used to looking at models of things, and want to be able to touch them and move them around. Manipulating them on a computer screen just doesn’t somehow seem quite right. But the ability to combine the virtual data, with the manipulation and movement of the real-life model gives us a unique and enhanced viewpoint, and can also allow us to visualise new buildings or exisiting buildings in new ways.

A particularly important consideration when creating AR content is to ensure that it looks as believable or ‘real’ as possible. The human eye is very good at noticing things that seem out of the ordinary or “don’t feel quite right”. One of the main ways to help with creating a believable AR experience  is to ensure the real-world occludes the virtual objects. That is the virtual content can be seen to move behind the real-world objects (such as the soldiers walking through the model gateway). Also it should be possible to interact with the real-world objects and have that affect the virtual content (such as touching one of the buildings and making the labels appear). This will become particularly important as I move into rolling the system out into a landscape instead of just a scale-model. As I augment the real world with virtual objects, those objects have to interact with the real-world as if they are part of it – otherwise too many Breaks in Presence will occur and the value of the AR content is diminished. An accurate 3D model of the real-world is quite a bit harder to create than that of a paper fort, but if I can pull it off, the results promise to be quite a bit more impressive…

 

ARK and Augmented Reality

Recently I have been working away in the Unity gaming engine using it to make some Augmented Reality applications for the iPhone and iPad. It is surprisingly successful and with at least 3 different ways of getting 3D content to overlay on the iOS video feed (Qualcomm, StringAR and UART) the workflow is more open than ever. I have been attempting to load 3D content at runtime, so that dynamic situations can be created as a result of user interaction – rather than having to have all of the resources (3D models, etc.) pre-loaded into the app. This not only saves on file size of the app, it also means that the app can pull real-time information and data that can be changed by many people at once. However, in order to do that I needed some kind of back-end database…

For those of you that know me, you will know that as well as doing my PhD I work on the development of the open-source archaeological database system known as the Archaeological Recording Kit (ARK). It seemed like a logical step to combine these two projects and use ARK as the back-end database. So that is what I went and did and at the same time created a rudimentary AR interface to ARK. The preliminary results can be seen in the video below:

This example uses the Qualcomm AR API, and ARK v1.0. Obviously at the moment it is marker-based AR (or at least image recognition based), the next task is to incorporate the iDevices’ gyroscope to enable the AR experience to continue even when the QR code is not visible.

TAG 2010

This weekend was the Annual Theoretical Archaeology Group (TAG) conference in Bristol. TAG is a great conference with many wide-ranging topics discussed, and has a tradition of being a platform from which a number of theories that may be conventionally considered ‘out-there’ are presented. It is also treated as the UK archaeological communities’ unofficial Christmas party. This year didn’t disappoint!

I was asked to contribute a paper to the CASPAR session on my work with Augmented Reality. The session was a mixture of papers concerning web-based approaches to archaeology, archaeology in television and radio and also virtual reality. As it was TAG I decided to inject quite a bit of theory into my paper, discussing the use of Turner’s Arc of Intentionality <image> to aid in analysing both real and virtual experiences. I attempted to explain how it is possible to use the Arc of Intentionality to get us a little closer to Husserl’s original ideas of phenomenology. That is, pulling apart an experience and analysing the individual parts.

The Arc of Intentionality (after Turner 2007)

One of the massive strengths of Augmented Reality in my mind is the simple fact of using the real world as a canvas – and only augmenting in a small amount of virtual objects, just the ones that you need. In a full Virtual Reality experience it is necessary to create the entire world as well – which leads to many many decisions about the experience, that we may not have enough information (or indeed desire) to make. Turner’s AoI then allows us to look at the augmented experience, acknowledge the parts of the virtual objects that ‘don’t quite feel right’ and either discard this part of the experience as unimportant and not something we are interested in – or to refine that particular element to get it closer to feeling like an authentic experience. After I delivered my paper a member of the audience pointed out that actually we may not even want the virtual objects to ‘feel right’ in every way. This is an excellent point and again fits nicely with Husserl’s ideas. If we are just interested in the impact of Hadrian’s Wall (for instance) as a barrier across the landscape – the perfect reconstruction of the mortar isn’t that important – what is important is the fact that you can’t see through it or over it. We need to take the parts of the experience we are interested in and concentrate just on them – rather than getting lost in the details.

I’m not sure I had quite enough time to go enough into the theory to get my point across properly, and I certainly don’t have enough room here but I recently submitted a paper that goes into a lot more detail on this subject, its currently being peer-reviewed, so fingers-crossed they will accept it!

Finally, a nice thing was that my live example of a simple marker-based AR project still seemed to produce the ‘wow’ factor. I guess this shows that AR is still relatively unknown (at least among archaeologists!) – it will be interesting to see where it all goes over the next year.

I have reproduced my slideshow below – and if you want to see the AR example, just print out this marker on a piece of A4 paper and point it at your webcam.

Slideshare: